Fix ISCP index consumer cancellation during DROP INDEX by jiangxinmeng1 · Pull Request #25733 · matrixorigin/matrixone

jiangxinmeng1 · 2026-07-15T02:29:27Z

What type of PR is this?

Which issue(s) this PR fixes:

What this PR does / why we need it:

This PR fixes DROP INDEX and other destructive ISCP-backed index DDL paths by
canceling and draining the active index consumer before index metadata and hidden
tables are removed.

Changes:

Add ISCP runtime tracking for running job consumers.
Fence dropped index job generations so new iterations skip them while DDL is
waiting for quiescence.
Cancel the consumer context and data retriever, then wait for the running
consumer to exit.
Route DROP INDEX, ALTER TABLE DROP INDEX, DROP TABLE, and DROP DATABASE
through the ISCP drain path before deleting metadata or hidden tables.
Use the existing global ISCP daemon task runner UUID from sys_daemon_task as
the single ISCP executor owner.
Add a queryservice-based cross-CN drain request so a DDL CN can ask the actual
ISCP runner CN to cancel and drain the local executor.
Keep the local executor fast path for single-CN and same-runner cases.
Fail closed when quiescence cannot be confirmed, including missing query client,
missing runner query address, or remote drain RPC failure.
Clean remote generation fences through transaction commit/rollback callbacks.
Make data retrievers reject/cleanup queued data after cancellation.
Add ctx-aware fault injection for WAIT and SLEEP.
Fix WAIT fault removal so blocked waiters are notified, including the
TriggerFaultWithContext path.
Add ISCP cancel failpoints for different consumer phases, including
per-index-name injection.
Add unit tests for runtime cancel, retriever cleanup, fault cancellation,
local drain, remote queryservice drain, remote fence cleanup, and DDL drain
behavior.

Test Matrix:

Area	Scenario	Result
Original reproduction	Reproduced the documented ISCP DROP INDEX cancel issue.
Fixed: DROP INDEX no longer leaves a background consumer running.
Single-process multi-CN	Started local multi-CN launch and issued DROP INDEX from
different CNs.	Passed for both directions.
Docker multi-process CN	Started `make dev-up` with separate `mo-cn1` and `mo-cn2`
containers. ISCP runner was on `mo-cn2`; DDL was issued from non-runner `mo-cn1`.
Passed.
Cross-CN DROP INDEX	Created async HNSW index from CN1 and dropped it from CN1 while
runner was CN2.	DROP INDEX succeeded through queryservice remote drain.
Cross-CN DROP TABLE	Created ISCP-backed HNSW table and dropped table from non-
runner CN.	DROP TABLE succeeded.
Cross-CN ALTER TABLE DROP INDEX	Created ISCP-backed HNSW table and ran ALTER TABLE
DROP INDEX from non-runner CN.	ALTER succeeded and index metadata was removed.
Cross-CN DROP DATABASE	Created ISCP-backed tables and dropped database from non-
runner CN.	DROP DATABASE succeeded.
In-flight remote consumer	Injected `fj/iscp/cancel/after-register-consumer` WAIT on
the runner CN, then issued DROP INDEX from another CN.	DROP waited before fault
removal, then completed after the runner consumer exited.
Cancel after consumer registration	`fj/iscp/cancel/after-register-consumer`.	DROP
waits for drain, then succeeds.
Cancel after worker submit	`fj/iscp/cancel/after-submit`.	Fenced job is skipped,
DROP succeeds.
Cancel during fanout	`fj/iscp/cancel/fanout-before-send`.	Canceled retriever
rejects data, DROP succeeds.
Cancel before HNSW update	`fj/iscp/cancel/long/hnsw-before-update`.	DROP returns
quickly.
Cancel before HNSW save	`fj/iscp/cancel/long/hnsw-before-save`.	DROP returns
quickly.
Cancel before watermark update	`fj/iscp/cancel/long/before-watermark`.	DROP
returns quickly.
Cancel before SQL execution	`fj/iscp/cancel/long/before-exec` with fulltext ASYNC.
DROP returns quickly.
Long wait via sleep	`fj/iscp/cancel/long/before-send:idx01` with sleep 30.	DROP
returns quickly after canceling the consumer context.
Iteration already failed before cancel	Injected iteration error before DROP.	Job
records the error path correctly; DROP still succeeds and sets `drop_at`.
Multiple jobs in one iteration	Two HNSW indexes on different columns, cancel only
`idx01`.	`idx01` removed; `idx02` preserved.
Fault cleanup	Listed fault points after tests.	No remaining fault points.
Goroutine cleanup	Checked pprof goroutines after fault/ISCP tests.	No stuck
cancel/scan/ISCP goroutines.
Unit tests	`go test -count=1 ./pkg/iscp ./pkg/sql/compile ./pkg/cnservice ./pkg/
queryservice/client` with local CGo flags.	Passed.
Build/package graph	`make build`.	Passed, including Go package graph check.
Static check	`git diff --check`.	Passed.

qodo-code-review · 2026-07-15T02:29:32Z

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

LeftHandCold

Re-reviewed the latest head (4ad3702). The previous unbounded generation-fence retention is materially improved: async finalization now removes the exact fence, and table GC has a fallback cleanup. The focused ISCP and compile tests pass. Two correctness blockers remain, though: the new runner lookup still dereferences a process-local registry and therefore cannot drain a separate CN process, and the successful-commit callback removes the fence before the cancel record is guaranteed to be visible in the runner in-memory state. Please address the inline findings and replace the stubbed runner-routing test with a real cross-process/RPC or durable-handshake test.

jiangxinmeng1 · 2026-07-16T06:58:06Z

This review is addressed by the latest changes.

The destructive DDL path no longer depends on a process-local executor lookup on
the DDL CN. We still use the global ISCP daemon task runner UUID from
sys_daemon_task/GetTaskRunner, but when that runner is remote, the DDL CN now
routes an ISCPDrainConsumer request through the existing queryservice to the
runner CN. The runner CN then performs CancelAndDrainJobConsumer against its
local executor before the DDL proceeds to drop metadata or hidden tables.

The generation fence lifetime is also handled for the remote path. The drain
request installs the fence on the runner CN, and the DDL transaction registers
commit/rollback callbacks that send a RemoveFenceOnly request back to the same
runner CN, so the remote fence is reclaimed after the DDL transaction reaches a
terminal outcome.

LeftHandCold

Re-reviewed current head dce285f. The queryservice RPC fixes the process-local cross-CN routing blocker. Two lifecycle blockers remain: the success fence is still removed before the drop logtail is visible, and remote rollback cleanup is a one-shot RPC whose failure can permanently fence a live job.

aptend

Re-reviewed the current head 1083581c61. The cross-CN drain routing and rollback recovery are materially improved, but the latest fence-lifetime change still leaves two generation-safety gaps, and the context-aware fault wait has an independent cross-waiter cancellation bug. Requesting changes for the following blockers:

[P1][generation safety] The success fence is removed before dropAt is installed in memory. addOrUpdateJob calls RemoveJobFence at pkg/iscp/executor.go:975-977, then parses the row, looks up the table, and only installs dropAt through AddOrUpdateSinker at line 1020. During that interval, TableEntry.getCandidate can observe the old entry with dropAt == 0 and no fence; both worker-side fence checks also see the fence removed, so the old generation can be admitted against hidden tables that the committed DDL has already dropped. I reproduced this deterministically by pausing addOrUpdateJob after fence removal but before the table lookup: getCandidate returned the dropped job. Please install the terminal in-memory state before removing the exact-generation fence, preserve the fence on parse/update failure, and add a regression covering this handoff.
[P1][generation safety] The fixed TTL reopens successful drops when logtail catch-up exceeds five minutes. CancelAndDrainJobConsumer assigns every fence DefaultRollbackFenceTTL, and isJobFencedLocked deletes it after expiration regardless of transaction outcome. The TTL solves permanent retention after failed rollback cleanup, but it also applies to a successfully committed DROP. If the runner cannot apply the drop_at logtail within five minutes, the old in-memory job becomes schedulable again and the original write-after-drop race returns. Please distinguish rollback/unknown-outcome recovery from a successful-drop fence, or renew/reconcile the lease against durable state so success remains fenced until dropAt is actually installed. Add a success-with-stalled-logtail regression.
[P2][fault correctness] Canceling one context-aware WAIT releases unrelated waiters. In pkg/util/fault/fault.go:232-244, each waiter's context watcher calls the shared condition's Broadcast(). With two TriggerFaultWithContext calls waiting on the same fault point, canceling only one context wakes both calls even though the second context is still live. A deterministic two-waiter regression fails immediately on this head. Please use per-waiter cancellation/registration or an equivalent predicate that does not turn one caller's cancellation into a global notify.

Validation on 1083581c61: focused tests for pkg/iscp, pkg/sql/compile, pkg/cnservice, pkg/queryservice/client, and pkg/util/fault pass; pkg/iscp and pkg/util/fault pass under -race, including focused -count=20 stress. The two deterministic regressions described above fail as expected and expose gaps not covered by the current tests. git diff --check passes.

jiangxinmeng1 · 2026-07-16T09:05:33Z

Re-reviewed the current head 1083581c61. The cross-CN drain routing and rollback recovery are materially improved, but the latest fence-lifetime change still leaves two generation-safety gaps, and the context-aware fault wait has an independent cross-waiter cancellation bug. Requesting changes for the following blockers:

[P1][generation safety] The success fence is removed before dropAt is installed in memory. addOrUpdateJob calls RemoveJobFence at pkg/iscp/executor.go:975-977, then parses the row, looks up the table, and only installs dropAt through AddOrUpdateSinker at line 1020. During that interval, TableEntry.getCandidate can observe the old entry with dropAt == 0 and no fence; both worker-side fence checks also see the fence removed, so the old generation can be admitted against hidden tables that the committed DDL has already dropped. I reproduced this deterministically by pausing addOrUpdateJob after fence removal but before the table lookup: getCandidate returned the dropped job. Please install the terminal in-memory state before removing the exact-generation fence, preserve the fence on parse/update failure, and add a regression covering this handoff.

[P1][generation safety] The fixed TTL reopens successful drops when logtail catch-up exceeds five minutes. CancelAndDrainJobConsumer assigns every fence DefaultRollbackFenceTTL, and isJobFencedLocked deletes it after expiration regardless of transaction outcome. The TTL solves permanent retention after failed rollback cleanup, but it also applies to a successfully committed DROP. If the runner cannot apply the drop_at logtail within five minutes, the old in-memory job becomes schedulable again and the original write-after-drop race returns. Please distinguish rollback/unknown-outcome recovery from a successful-drop fence, or renew/reconcile the lease against durable state so success remains fenced until dropAt is actually installed. Add a success-with-stalled-logtail regression.

[P2][fault correctness] Canceling one context-aware WAIT releases unrelated waiters. In pkg/util/fault/fault.go:232-244, each waiter's context watcher calls the shared condition's Broadcast(). With two TriggerFaultWithContext calls waiting on the same fault point, canceling only one context wakes both calls even though the second context is still live. A deterministic two-waiter regression fails immediately on this head. Please use per-waiter cancellation/registration or an equivalent predicate that does not turn one caller's cancellation into a global notify.

Validation on 1083581c61: focused tests for pkg/iscp, pkg/sql/compile, pkg/cnservice, pkg/queryservice/client, and pkg/util/fault pass; pkg/iscp and pkg/util/fault pass under -race, including focused -count=20 stress. The two deterministic regressions described above fail as expected and expose gaps not covered by the current tests. git diff --check passes.

Thanks for calling this out. I intentionally did not change the 5-minute fence TTL
semantics in this revision.

The fence is installed only after the foreground DROP has synchronously routed to the
ISCP runner and drained the currently running consumer. After that point, the remaining
window is the runner applying the committed drop_at logtail into its in-memory job
state. Under normal operation that catch-up should complete well within the existing
rollback fence TTL; the TTL is mainly a recovery guard so a rolled-back DROP whose
rollback cleanup RPC is lost does not leave a live job fenced forever.

Making successful DROP fences non-expiring until drop_at is applied would require
introducing an additional commit-outcome/generation state, or another reliable cross-CN
handoff, so rollback cleanup remains recoverable without permanently fencing a live
job. That is a larger lifecycle change than I want to add in this PR. For this patch I
kept the existing TTL behavior, and fixed the concrete generation handoff bug by moving
fence removal until after drop_at has been installed in memory and preserving the
fence on parse/update failure.

The added regression covers this handoff: after addOrUpdateJob installs drop_at and
removes the fence, getCandidate still does not return the dropped job; if parsing/
updating the dropped job fails, the generation fence is preserved.

LeftHandCold

Re-reviewed current head a08c234. The previous commit-to-logtail cleanup race and permanent rollback-fence leak are addressed, but the new fixed fence TTL can expire while a valid explicit DDL transaction is still open, reopening the consumer/hidden-table race this PR is intended to close.

aptend

Reviewed current head 70aba40f90. The latest change only extends the in-memory fence TTL from 5 to 30 minutes; it does not close the generation/terminal handoff gaps below. Requesting changes for three blockers:

[P1][generation safety] The fence is lost when the ISCP runner executor is replaced. fencedJobs is process-local and every new ISCPTaskExecutor starts with an empty map. If the runner CN restarts or the daemon-task lease migrates while the DROP transaction is still open, the replacement executor replays the still-live durable job with no fence and can admit it against the hidden tables again. The lease cannot repair this: renewal is pinned to the originally resolved query address, RenewJobFence silently does nothing when the key is absent, and the RPC handler still returns success. I reproduced this with a deterministic executor-replacement regression: the old executor is fenced, the replacement receives the current renewal call, and replacement.IsJobFenced(key) is false. Please make the fence durable/reconstructible across runner generations (or explicitly hand it off/re-resolve and install it), make renewal fail when the fence is missing, and add a runner restart/lease-migration regression while the DDL transaction is unresolved.
[P1][correctness] Renewal stops at commit entry, before either commit or the drop_at logtail handoff is complete. The non-cost CommitEvent invokes lease.Stop(), but txnOperator.Commit emits that event before doWrite. Therefore a slow/unknown commit or delayed logtail can outlive the remaining TTL; IsJobFenced then deletes the fence while the in-memory job still has dropAt == 0, reopening the original consumer/hidden-table race. Increasing the TTL to 30 minutes only delays this failure. The new test explicitly expects the fence to expire after commit starts, which codifies the unsafe contract. Remote renewal failures have the same problem: they are only logged, so an unresolved DDL silently becomes unfenced after the TTL. Please keep a successful-drop fence until addOrUpdateJob installs dropAt, remove/recover it only for rollback or a definitely failed commit, reconcile unknown outcomes, and make renewal failure fail closed. Add short injected-TTL regressions for commit-before-logtail, stalled logtail, and failed renewal.
[P2][build reproducibility] proto/query.proto and query.pb.go disagree. The generated Go request contains and the implementation uses RenewFenceOnly field 6, but ISCPDrainConsumerRequest in proto/query.proto ends at RemoveFenceOnly = 5. Regenerating protobufs removes the field and breaks the two call sites. Please add bool RenewFenceOnly = 6; to the source proto and regenerate query.pb.go with the repository target.

Validation: existing focused tests for pkg/iscp, the drain-related pkg/sql/compile and pkg/cnservice cases, and pkg/util/fault -race pass; git diff --check passes. The deterministic runner-replacement regression described above fails on this head.

jiangxinmeng1 · 2026-07-16T10:03:48Z

I fixed the proto source mismatch by adding RenewFenceOnly = 6 to proto/ query.proto.

For the durable fence suggestion, I agree that a purely in-memory fence cannot survive
runner restart or lease migration. However, I do not think making this fence durable is
the right fix for this PR. A durable fence needs a full lifecycle protocol for commit,
rollback, unknown commit outcome, executor recovery, and cleanup. If any cleanup path
is missed, the stale durable fence can permanently suppress a still-valid ISCP job and
leave the index stale. In this case, a leftover fence is more dangerous than the
transient in-memory fence loss, because it can silently stop future maintenance for a
live index until manual intervention.

The current patch keeps the fence process-local and transaction-scoped, with renewal
while the DDL transaction is unresolved. This avoids introducing a new durable state
machine with incomplete cleanup guarantees. I agree this means runner restart or lease
migration during an unresolved DDL transaction is not fully covered by the in-memory
fence design; covering that correctly should be handled with a separate, explicit
cross-runner lifecycle design rather than by persisting this map opportunistically

aptend

There is still a generation-safety gap in the cancellation fence.

The fence is stored only in the current executor's in-memory fencedJobs map. If the ISCP executor restarts or the daemon-task lease migrates while the DROP transaction is unresolved, the replacement executor starts with an empty fence map and can rebuild/admit the still-visible job (dropAt is not committed yet). The local lease also keeps renewing the old executor pointer.

The remote path has the same safety hole after restart or expiry: RenewJobFence silently does nothing when the exact fence is absent, while handleISCPDrainConsumer returns success. The DDL side therefore believes the fence is renewed even though the consumer can run again, reopening the DROP INDEX/consumer race this PR is intended to close.

Please make the fence durable or executor-generation/lease-owner aware, and require renewal to prove that the exact fence still exists (otherwise re-resolve the runner and re-drain/reinstall it before acknowledging success). The DDL must not commit if quiescence can no longer be proven. Please also add coverage for executor restart/task migration during an unresolved DDL transaction and renewal after the fence has expired or disappeared.

LeftHandCold

Found one remaining correctness issue in the remote fence renewal. The lease can still expire while a renewal RPC is in flight, and a late renewal is then silently acknowledged after the key has already been deleted, permanently losing the protection during an open DDL transaction. Requesting changes for the inline issue. I rechecked the consumer cancellation/drain, fan-out refcounting, rollback cleanup, and committed drop-logtail handoff; those paths look closed. The affected package tests pass.

jiangxinmeng1 · 2026-07-17T08:56:54Z

There is still a generation-safety gap in the cancellation fence.

The fence is stored only in the current executor's in-memory fencedJobs map. If the ISCP executor restarts or the daemon-task lease migrates while the DROP transaction is unresolved, the replacement executor starts with an empty fence map and can rebuild/admit the still-visible job (dropAt is not committed yet). The local lease also keeps renewing the old executor pointer.

The remote path has the same safety hole after restart or expiry: RenewJobFence silently does nothing when the exact fence is absent, while handleISCPDrainConsumer returns success. The DDL side therefore believes the fence is renewed even though the consumer can run again, reopening the DROP INDEX/consumer race this PR is intended to close.

Please make the fence durable or executor-generation/lease-owner aware, and require renewal to prove that the exact fence still exists (otherwise re-resolve the runner and re-drain/reinstall it before acknowledging success). The DDL must not commit if quiescence can no longer be proven. Please also add coverage for executor restart/task migration during an unresolved DDL transaction and renewal after the fence has expired or disappeared.

I agree that an in-memory fence cannot provide a hard guarantee across runner restart
or daemon-task lease migration while the DDL transaction is still unresolved. I also
agree that a non-durable re-resolve/re-drain scheme has its own risks: it adds more
cross-CN retry logic inside an open DDL transaction, has to reason about unknown commit
outcomes, and can still fail to prove quiescence under partitions or runner churn.

For this PR, I intentionally kept the fence process-local and transaction-scoped. The
default TTL is now 30 minutes and the lease is renewed while the DDL transaction
remains unresolved, so the normal long-transaction case is covered without adding a new
durable state machine or a complex cross-runner handoff protocol. Persisting the fence
is also risky: a stale durable fence can permanently suppress a still-valid ISCP job
and leave the index stale until manual intervention.

Given that tradeoff, I would prefer to keep this PR scoped to the active-runner
lifecycle: cancel and drain the current consumer, keep the in-memory generation fence
alive during the unresolved DDL transaction, remove it on rollback, and clear it after
the committed drop_at is installed. Runner crash or daemon-task migration during an
unresolved destructive DDL is a narrower availability/failure-domain case that should
be handled by a separate lifecycle design rather than by opportunistically persisting
this fence map in this PR.

@aptend

aptend · 2026-07-17T09:00:26Z

@jiangxinmeng1 Thanks, I agree that persisting the fence is not necessarily the right scope for this PR, and runner restart/daemon-task lease migration can be tracked as a separate lifecycle design.

However, the current implementation still does not guarantee the active-runner lifecycle described here. No restart or migration is required: a renewal RPC starts at TTL/2 but may run for a full TTL. The existing fence can therefore expire while renewal is still in flight. Once IsJobFenced deletes the expired key, RenewJobFence silently no-ops and handleISCPDrainConsumer still returns success. The DDL side consequently does not fail closed, and the consumer can be admitted again while the transaction remains unresolved.

This is the same active-runner case described in #25733 (comment), and does not require a durable fence. Please make renewal finish within the remaining lease with a safety margin, return an explicit error for a missing/expired fence, and ensure loss of the lease prevents the DDL from committing. With that closed, I am fine with documenting restart/migration as a separate follow-up.

…umer-cancel # Conflicts: # pkg/pb/query/query.pb.go

LeftHandCold

Re-reviewed the latest head fc114288d7. The remaining fence-renewal failure mode can re-expose an intermittent DDL deadlock/retry window, but it does not establish a committed data-correctness failure: the initial consumer drain closes the active race, the destructive DDL changes are transactional, and consumer result updates do not clear drop_at. Treating renewal as best-effort avoids turning a transient control-plane failure into a guaranteed rollback of an otherwise valid transaction. I do not see a remaining blocker on this head.

jiangxinmeng1 · 2026-07-17T09:49:36Z

@jiangxinmeng1 Thanks, I agree that persisting the fence is not necessarily the right scope for this PR, and runner restart/daemon-task lease migration can be tracked as a separate lifecycle design.

However, the current implementation still does not guarantee the active-runner lifecycle described here. No restart or migration is required: a renewal RPC starts at TTL/2 but may run for a full TTL. The existing fence can therefore expire while renewal is still in flight. Once IsJobFenced deletes the expired key, RenewJobFence silently no-ops and handleISCPDrainConsumer still returns success. The DDL side consequently does not fail closed, and the consumer can be admitted again while the transaction remains unresolved.

This is the same active-runner case described in #25733 (comment), and does not require a durable fence. Please make renewal finish within the remaining lease with a safety margin, return an explicit error for a missing/expired fence, and ensure loss of the lease prevents the DDL from committing. With that closed, I am fine with documenting restart/migration as a separate follow-up.

@aptend
The original issue already demonstrates that, without a confirmed ISCP quiescence
fence, the DDL path does not silently succeed in the observed race. It fails with
cannot confirm ISCP consumer quiescence / DDL conflict instead of completing hidden-
table removal under an active consumer. So losing the renewal does not necessarily mean
the DDL must be force-aborted by the lease layer itself; the underlying DDL/lock path
can still fail the statement.

That said, I agree the renewal ACK semantics must not be misleading. I have changed
RenewJobFence / RenewFenceOnly so a missing or expired exact fence returns an
explicit error, and I bounded the renewal RPC timeout to complete within the lease
window. With that change, renewal loss is visible instead of being acknowledged as
success.

I would prefer not to add another pre-commit abort path in this PR unless we can show
that the DDL can silently commit hidden-table removal after the active-runner fence is
lost. The currently observed failure mode is an explicit DDL error, which is acceptable
for this best-effort in-memory fence.

gouhongshen

Codex automated review

The fence-renewal failure path is not enforced, so a runner restart during an open DROP INDEX transaction can re-enable the consumer before the drop commits.

P1 - Abort or otherwise preserve quiescence when fence renewal fails (pkg/sql/compile/iscp_util.go:365)

Fence state is in-memory only. If the ISCP runner restarts (or the fence expires) while the DDL transaction remains open, RenewJobFence correctly returns false, but this goroutine only logs the error and continues. The replacement executor has no fence and RegisterRunningConsumer accepts the still-active, uncommitted job, allowing it to resume writes to the index being dropped. The new handler error for a missing fence is therefore never propagated to the DDL transaction. The renewal failure must keep the job fenced or cause the DROP transaction to fail/abort; logging is insufficient.

fix: cancel ISCP index consumers on drop index

d1f23cc

jiangxinmeng1 requested review from LeftHandCold, XuPeng-SH and aunjgr as code owners July 15, 2026 02:29

jiangxinmeng1 had a problem deploying to ci July 15, 2026 02:29 — with GitHub Actions Error

jiangxinmeng1 temporarily deployed to ci July 15, 2026 02:29 — with GitHub Actions Inactive

jiangxinmeng1 had a problem deploying to ci July 15, 2026 02:29 — with GitHub Actions Error

jiangxinmeng1 had a problem deploying to ci July 15, 2026 02:30 — with GitHub Actions Error

Merge branch 'main' into fix-iscp-index-consumer-cancel

38d635b

mergify Bot added the kind/bug Something isn't working label Jul 15, 2026

mergify Bot temporarily deployed to ci July 15, 2026 02:30 Inactive

mergify Bot had a problem deploying to ci July 15, 2026 02:30 Failure

mergify Bot temporarily deployed to ci July 15, 2026 02:31 Inactive

mergify Bot had a problem deploying to ci July 15, 2026 02:31 Failure

matrix-meow added the size/XL Denotes a PR that changes [1000, 1999] lines label Jul 15, 2026

test: cover ISCP cancel error paths

fcce6f3

jiangxinmeng1 temporarily deployed to ci July 15, 2026 03:49 — with GitHub Actions Inactive

jiangxinmeng1 temporarily deployed to ci July 15, 2026 03:50 — with GitHub Actions Inactive

jiangxinmeng1 had a problem deploying to ci July 16, 2026 06:09 — with GitHub Actions Error

jiangxinmeng1 requested a review from LeftHandCold July 16, 2026 06:09

LeftHandCold requested changes Jul 16, 2026

View reviewed changes

Comment thread pkg/sql/compile/iscp_util.go Outdated

Comment thread pkg/sql/compile/iscp_util.go Outdated

fix: route iscp drain to runner cn

dce285f

jiangxinmeng1 requested review from aptend and fengttt as code owners July 16, 2026 06:59

LeftHandCold requested changes Jul 16, 2026

View reviewed changes

Comment thread pkg/sql/compile/iscp_util.go Outdated

Comment thread pkg/sql/compile/iscp_util.go Outdated

fix: retain iscp fences until drop logtail

1083581

aptend requested changes Jul 16, 2026

View reviewed changes

jiangxinmeng1 added 2 commits July 16, 2026 16:53

fix: query iscp runner from system account

195e9f3

fix: close iscp fence handoff window

a08c234

LeftHandCold requested changes Jul 16, 2026

View reviewed changes

Comment thread pkg/iscp/runtime_cancel.go Outdated

jiangxinmeng1 added 2 commits July 16, 2026 17:40

fix: renew iscp fences during open ddl txn

6f143db

fix: extend iscp fence lease timeout

70aba40

aptend requested changes Jul 16, 2026

View reviewed changes

fix: add iscp renew fence field to proto

0c3b673

fengttt approved these changes Jul 16, 2026

View reviewed changes

aunjgr approved these changes Jul 17, 2026

View reviewed changes

aptend requested changes Jul 17, 2026

View reviewed changes

LeftHandCold requested changes Jul 17, 2026

View reviewed changes

Comment thread pkg/sql/compile/iscp_util.go

fix: fail missing iscp fence renewal

992916e

Merge remote-tracking branch 'upstream/main' into fix-iscp-index-cons…

fc11428

…umer-cancel # Conflicts: # pkg/pb/query/query.pb.go

LeftHandCold approved these changes Jul 17, 2026

View reviewed changes

gouhongshen requested changes Jul 18, 2026

View reviewed changes

Uh oh!

Conversation

jiangxinmeng1 commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What type of PR is this?

Which issue(s) this PR fixes:

What this PR does / why we need it:

Uh oh!

qodo-code-review Bot commented Jul 15, 2026

Qodo reviews are paused for this user.

Uh oh!

LeftHandCold left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jiangxinmeng1 commented Jul 16, 2026

Uh oh!

LeftHandCold left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

aptend left a comment

Choose a reason for hiding this comment

Uh oh!

jiangxinmeng1 commented Jul 16, 2026

Uh oh!

LeftHandCold left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

aptend left a comment

Choose a reason for hiding this comment

Uh oh!

jiangxinmeng1 commented Jul 16, 2026

Uh oh!

aptend left a comment

Choose a reason for hiding this comment

Uh oh!

LeftHandCold left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jiangxinmeng1 commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aptend commented Jul 17, 2026

Uh oh!

LeftHandCold left a comment

Choose a reason for hiding this comment

Uh oh!

jiangxinmeng1 commented Jul 17, 2026

Uh oh!

gouhongshen left a comment

Choose a reason for hiding this comment

Codex automated review

P1 - Abort or otherwise preserve quiescence when fence renewal fails (pkg/sql/compile/iscp_util.go:365)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

jiangxinmeng1 commented Jul 15, 2026 •

edited

Loading

jiangxinmeng1 commented Jul 17, 2026 •

edited

Loading